Visual speech segmentation and speaker recognition for transcription of TV news

This paper is about a method for visual segmentation of TV news. The TV news shows are segmented according to the visual stream from the video TV recordings in this method. Human faces are found in the single visual segments with the help of the fast algorithm for face detection. The found faces are compared with the visual GMMs, that have been trained from the video picture of the single broadcasters (anchors) from the TV news. The single visual segments, where the faces of the broadcasters have been found and recognized, have been compared with the acoustic segments from the acoustic segmentation. The speaker adapted HMMs have been used for speech recognition of these acoustic segments. The recognition rate is better for the use of this speaker-adapted HMMs compared to the use of the speaker independent HMMs. It is possible to use the methods for the speaker identification and verification from the acoustic signal in the acoustic segments. The results from the visual speaker identification will be better for smaller number of speakers and for the use of the video recordings of TV news with a lot of noise in the acoustic signal.